Using Probabilistic Ontologies for Video Exploration

Video data is being collected at alarming rates and yet there exists no comprehensive forensic toolset that enables the analyst to quickly examine video in the context of the massive collections. This research builds a System that studies video at a semantic level by means of a joint solution to semantic entity extraction, entity-entity relationship extraction, and dynamic event recognition. The working of the System is grounded in formal ontology. This ontology is jointly induced from the data and established by the human domain experts (i.e., interactive machine learning). Specifically, we implement a Multi Entity Bayesian Network (a form of a probabilistic ontology); we test our System on two-on-two basketball game videos, and our results demonstrate state of the art detection rates on activities like passing the ball, and shooting, consequently promising that the presented methodology is an encouraging direction for semantically rich video analysis.